From Deterministic to Generative: Multi-Modal Stochastic RNNs for Video Captioning

Video captioning in essential is a complex natural process, which is affected by various uncertainties stemming from video content, subjective judgment, etc. In this paper we build on the recent progress in using encoder-decoder framework for video captioning and address what we find to be a critical deficiency of the existing methods, that most of the decoders propagate deterministic hidden states. Such complex uncertainty cannot be modeled efficiently by the deterministic models. In this paper, we propose a generative approach, referred to as multi-modal stochastic RNNs networks (MS-RNN), which models the uncertainty observed in the data using latent stochastic variables. Therefore, MS-RNN can improve the performance of video captioning, and generate multiple sentences to describe a video considering different random factors. Specifically, a multi-modal LSTM (M-LSTM) is first proposed to interact with both visual and textual features to capture a high-level representation. Then, a backward stochastic LSTM (S-LSTM) is proposed to support uncertainty propagation by introducing latent variables. Experimental results on the challenging datasets MSVD and MSR-VTT show that our proposed MS-RNN approach outperforms the state-of-the-art video captioning benchmarks

A Spatiotemporal Volumetric Interpolation Network for 4D Dynamic Medical Image

Dynamic medical imaging is usually limited in application due to the large radiation doses and longer image scanning and reconstruction times. Existing methods attempt to reduce the dynamic sequence by interpolating the volumes between the acquired image volumes. However, these methods are limited to either 2D images and/or are unable to support large variations in the motion between the image volume sequences. In this paper, we present a spatiotemporal volumetric interpolation network (SVIN) designed for 4D dynamic medical images. SVIN introduces dual networks: first is the spatiotemporal motion network that leverages the 3D convolutional neural network (CNN) for unsupervised parametric volumetric registration to derive spatiotemporal motion field from two-image volumes; the second is the sequential volumetric interpolation network, which uses the derived motion field to interpolate image volumes, together with a new regression-based module to characterize the periodic motion cycles in functional organ structures. We also introduce an adaptive multi-scale architecture to capture the volumetric large anatomy motions. Experimental results demonstrated that our SVIN outperformed state-of-the-art temporal medical interpolation methods and natural video interpolation methods that have been extended to support volumetric images. Our ablation study further exemplified that our motion network was able to better represent the large functional motion compared with the state-of-the-art unsupervised medical registration methods.Comment: 10 pages, 8 figures, Conference on Computer Vision and Pattern Recognition (CVPR) 202

The Improvement of Hyperglycemia after RYGB Surgery in Diabetic Rats Is Related to Elevated Hypothalamus GLP-1 Receptor Expression

Objectives. This study aimed to explore the expression of GLP-1 receptor in hypothalamus and gastrointestinal tissues after Roux-en-Y gastric bypass (RYGB) surgery in diabetic rats. Methods. Male 12-week-old Wistar rats (control) and Goto-Kakizaki rats (diabetic) were randomly divided into two groups, respectively: control sham surgery group (C), control RYGB group (C + R), diabetic sham surgery group (D), and diabetic RYGB group (D + R). Body weight and blood glucose were monitored before and after surgery every week. Eight weeks after surgery, all rats were sacrificed and the serum fasting GLP-1 concentrations were measured by ELISA. GLP-1R and DPP-4 expression in hypothalamus and ileum were measured by RT-PCR. Results. The body weight and fasting/random blood glucose in the D + R group decreased significantly compared with the D group (P<0.05). Serum GLP-1 levels in diabetic rats treated with RYGB were higher than the corresponding sham surgery rats. The expression of GLP-1R of hypothalamus in RYGB-treated diabetic rats was significantly higher than those of the sham surgery diabetic rats and both control group rats (P<0.05). We found a negative correlation between hypothalamus GLP-1R mRNA and blood glucose level. No significant difference was seen in ileum GLP-1R and DPP-4 expression among all groups. Conclusions. RYGB efficiently promoted serum GLP-1 levels and the expression of GLP-1 receptor in the hypothalamus in diabetic rats. These data suggest that the hypothalamus GLP-1R may play an important role in the GLP-1 system for improving glucose homeostasis after reconstruction of the gastrointestinal tract

Ginsenoside Rh1 Improves the Effect of Dexamethasone on Autoantibodies Production and Lymphoproliferation in MRL/lpr Mice

Ginsenoside Rh1 is able to upregulate glucocorticoid receptor (GR) level, suggesting Rh1 may improve glucocorticoid efficacy in hormone-dependent diseases. Therefore, we investigated whether Rh1 could enhance the effect of dexamethasone (Dex) in the treatment of MRL/lpr mice. MRL/lpr mice were treated with vehicle, Dex, Rh1, or Dex + Rh1 for 4 weeks. Dex significantly reduced the proteinuria and anti-dsDNA and anti-ANA autoantibodies. The levels of proteinuria and anti-dsDNA and anti-ANA autoantibodies were further decreased in Dex + Rh1 group. Dex, Rh1, or Dex + Rh1 did not alter the proportion of CD4+ splenic lymphocytes, whereas the proportion of CD8+ splenic lymphocytes was significantly increased in Dex and Dex + Rh1 groups. Dex + Rh1 significantly decreased the ratio of CD4+/CD8+ splenic lymphocytes compared with control. Con A-induced CD4+ splenic lymphocytes proliferation was increased in Dex-treated mice and was inhibited in Dex + Rh1-treated mice. Th1 cytokine IFN-γ mRNA was suppressed and Th2 cytokine IL-4 mRNA was increased by Dex. The effect of Dex on IFN-γ and IL-4 mRNA was enhanced by Rh1. In conclusion, our data suggest that Rh1 may enhance the effect of Dex in the treatment of MRL/lpr mice through regulating CD4+ T cells activation and Th1/Th2 balance

Relationship between thick or greasy tongue-coating microbiota and tongue diagnosis in patients with primary liver cancer

Tongue diagnosis is a unique aspect of traditional Chinese medicine for diagnosing diseases before determining proper means of treatment, but it also has the disadvantage of relying on the subjective experience of medical practitioners and lack objective basis. The purpose of this article is to elucidate tongue-coating microbiota and metabolic differences in primary liver cancer (PLC) patients with thick or greasy tongue coatings. Tongue-coating samples were analyzed in 60 PLC patients (30 PLC with thick or greasy tongue-coating patients and 30 PLC with tongue-coating neither thick nor greasy) and 25 healthy controls (HC) using 16S rRNA gene sequencing technology. As compared to healthy individuals, tongue coatings of patients with PLC had elevated levels of Firmicutes and Actinobacteria. The abundance of Fusobacteria, SR1_Absconditabacteria_, and Spirochaete were higher in tongue coatings of healthy controls compared to samples in patients with PLC. In addition to site-specific differences, higher abundances of Fusobacteria and Actinobacteria were observed in thick or greasy tongue-coating patients as compared to non-thick and greasy tongue-coating patients. The inferred metagenomic pathways enriched in the PLC tongue-coating patients were mainly those involved in replication, recombination, and repair of protein. We also identify a tongue-coating microbiome signature to discriminate HC and PLC, including 15 variables on genus level. The prediction performance of the signature showed well in the training and validation cohorts. This research illustrates specific clinical features and bacterial structures in PLC patients with different tongue coatings, which facilitates understanding of the traditional tongue diagnosis

A global product of fine-scale urban building height based on spaceborne lidar

Characterizing urban environments with broad coverages and high precision is more important than ever for achieving the UN's Sustainable Development Goals (SDGs) as half of the world's populations are living in cities. Urban building height as a fundamental 3D urban structural feature has far-reaching applications. However, so far, producing readily available datasets of recent urban building heights with fine spatial resolutions and global coverages remains a challenging task. Here, we provide an up-to-date global product of urban building heights based on a fine grid size of 150 m around 2020 by combining the spaceborne lidar instrument of GEDI and multi-sourced data including remotely sensed images (i.e., Landsat-8, Sentinel-2, and Sentinel-1) and topographic data. Our results revealed that the estimated method of building height samples based on the GEDI data was effective with 0.78 of Pearson's r and 3.67 m of RMSE in comparison to the reference data. The mapping product also demonstrated good performance as indicated by its strong correlation with the reference data (i.e., Pearson's r = 0.71, RMSE = 4.60 m). Compared with the currently existing products, our global urban building height map holds the ability to provide a higher spatial resolution (i.e., 150 m) with a great level of inherent details about the spatial heterogeneity and flexibility of updating using the GEDI samples as inputs. This work will boost future urban studies across many fields including climate, environmental, ecological, and social sciences